UNCLASSIFIED 


Defense  Technical  Information  Center 
Compilation  Part  Notice 

ADP014628 

TITLE:  Evaluation  of  Performance  Reliability  Using  Regression  Models 
DISTRIBUTION:  Approved  for  public  release,  distribution  unlimited 


This  paper  is  part  of  the  following  report: 

TITLE:  Proceedings  of  the  Eighth  Conference  on  the  Design  of 
Experiments  in  Army  Research  Development  and  Testing 

To  order  the  complete  compilation  report,  use:  ADA4 19759 

The  component  part  is  provided  here  to  allow  users  access  to  individually  authored  sections 
of  proceedings,  annals,  symposia,  etc.  However,  the  component  should  be  considered  within 
the  context  of  the  overall  compilation  report  and  not  as  a stand-alone  technical  report. 

The  following  component  part  numbers  comprise  the  compilation  report: 

ADPO 14598  thru  ADPO 14630 


UNCLASSIFIED 
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Seymour  K.  Einbinder,  Pic atinny  Arsenal 

and 

Ingram  Olkin,  Stanford  University 

0.  ABSTRAC T.  Performance  reliability  is  the  probability  that  a 
weapon  will  perform  its  prescribed  function  under  given  conditions  of  en- 
vironment at  some  particular  time.  Performance  reliability  models  are 
defined  for  continuous  performance  response  variables.  Procedures  are 
then  described  for  the  evaluation  of  reliability  with  emphasis  on  the  appli- 
cation of  univariate  and  multivariate  regression  analysis  to  single  and 
multiple  continuous  response  variables,  respectively.  Point  and  confi- 
dence interval  estimation  methods  for  performance  reliability  are  dis- 
cussed, and  a sample  problem  is  presented  illustrating  some  of  the  basic 
concepts  and  results. 

1.  INTRODUCTION.  A major  problem  during  the  research  and  de- 
velopment of  a weapon  or  warhead  is  the  assurance  of  high  functioning  re- 
liability of  the  final  prototype  design.  The  reliability  concepts  and  eval- 
uation methods  to  be  described  are  general  and  are  applicable  to  a wide 
variety  of  systems  and  components. 

A weapon  during  its  lifetime  may  be  subjected  to  many  environ- 
mental factors  or  stresses  such  as  temperature,  vibration,  acceleration, 
rough  handling,  etc.  In  addition,  the  stresses  may  be  encountered  singly, 
simultaneously  or  in  sequence.  The  problem  of  testing  and  estimating  re- 
liability is  of  importance  to  the  weapon  developer  in  order  to  assure  the 
user  of  a reliable  weapon  for  use  in  any  potential  combat  situation. 

The  establishment  of  high  reliability  with  a high  level  of  confidence 
generally  requires  the  testing  of  larger  numbers  of  items  than  are  usually 
available  during  a development  program  for  a complex  and  expensive  item. 
Thus,  it  is  generally  necessary  to  obtain  the  most  information  with  a 
minimum  number  of  samples  and  tests.  Improved  and  more  efficient 
statistical  methods  are  required  in  many  cases  to  solve  the  reliability 
estimation  problem. 

Before  solutions  to  a problem  can  be  obtained,  it  is  important  to 
delineate  the  problem  so  that  a representative  mathematical  model  can 
be  developed.  Obviously,  any  solutions  obtained  can  be  no  better  than 
the  underlying  mathematical  model  which  is  assumed  as  a prototype  of 
the  problem. 
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In  a previous  paper  , the  emphasis  was  on  test-to-failure  and 
stress  vs.  strength  analyses  for  single  and  multiple  environments  (stresses). 
The  present  paper  is  concerned  mainly  with  reliability  in  the  case  of  a 
continuous  regression  response  surface,  and  thereby  is  an  extension  of  3 . 
Methods  of  point  and  interval  estimation  for  the  univariate  and  multivariate 
regression  problems  are  discussed. 

2.  RELIABILITY  CONCEPTS.  Reliability  of  a weapon  may  be  defined 
as  the  probability  of  a successful  functioning  under  required  conditions  of 
the  environment  at  some  particular  time.  Successful  functioning  might 
require  the  successful  operation  of  several  or  all  components  of  a system. 

The  outputs  or  responses  of  each  component  may  be  attribute  or  continuous  . 

In  the  case  of  the  continuous  response,  success  may  require  that  the  response 
lie  within  certain  limits  (possibly  specification  limits), 

To  illustrate  these  concepts,  a hypothetical  shaped  charge  warhead 
section  for  a missile  will  be  used  as  an  example.  Successful  functioning 
of  the  warhead  section  may  require  that  the  S and  A (Safety  and  Arming 
Device)  must  arm  and  detonate  the  warhead  on  target  impact,  and  the 
warhead  must  then  penetrate  at  least  a specified  distance  into  an  armor 
plate  target.  This  example  could  be  further  complicated  by  specifying 
arming  limits  for  the  S and  A.  Failure  of  the  warhead  section  may 
occur  in  two  fundamental  ways; 

(1)  a complete  dud  or  catastrophic  failure  may  result  such  that  no 
warhead  detonation  takes  place,  or, 

(2)  the  warhead  explosive  train  may  be  initiated  but  the  armor  pene- 
tration requirement  may  not  be  met. 

The  reliability  of  the  warhead  section  is  given  by 


R = (l 


■p  \ n 
D WHD? 


where  R is  the  overall  warhead  section  reliability,  is  the  pro- 
bability of  a dud  or  catastrophic  failure,  and  is  the  conditional 

probability  that  the  warhead  exceeds  the  specified  performance  require- 
ment. The  latter  will  be  referred  to  as  the  performance  reliability  and 
is  of  prime  concern  in  this  paper.  The  dud  probability  can  be  broken  down 
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further  according  to  various  components,  In  general  it  is  necessary  to 
evaluate  the  dud  or  catastrophic  failures  separately  from  the  performance 
failures  since  they  do  not  have  the  same  distribution  and  are  mutually  ex- 
clusive. Dud  failures,  being  attribute,  normally  require  larger  sample 
sizes  for  evaluation  with  the  same  precision  and  confidence  levels  as 
would  performance  reliability  based  on  continuous  variables. 

The  remainder  of  this  paper  will  be  concerned  with  the  evaluation  of 
the  performance  reliability,  R^.^. 

3.  MATHEMATICAL  MODELS.  In  this  section  we  define  the  mathe-  ' 
matical  models  upon  which  subsequent  analysis  is  based.  Univariate  and 
multivariate  responses  and  single  and  multiple  stresses  are  considered. 
Estimation  procedures  are  described  in  the  subsequent  sections. 

3.1  Univariate  Response.  We  have  defined  performance  reliability  as 
the  probability  that  a continuous  performance  variable  lies  within  certain 
specified  limits.  Thus,  it  may  be  required  that  the  arming  time  for  an  S 
and  A Device  be  greater  than  some  minimum  time  required  for  safety.  The 
performance  variable  or  response  may  be  thought  of  as  a dependent  variable 
which  is  a function  of  one  or  more  environments  or  stresses  which  are  the 
independent  variables.  In  general,  this  functional  relationship  is  unknown; 
however,  we  can  approximate  the  response  function  over  small  regions  of 
the  function  space  by  linear  regression  methods.  G.enerally,  we  are  con- 
cerned with  the  reliability  under  some  critical  stress  conditions.  These 
conditions  will  be  referred  to  as  a critical  point  or  critical  reliability 
boundary.  The  regression  experiment  is  designed  to  provide  the  best  infor- 
mation in  the  vicinity  of  this  point.  Figure  1 illustrates  the  experimental 
design  in  general  terms  for  the  univariate  case,  (x  . . . , x ) represent 

the  applied  stresses  or  environments  such  as  temperature,  vibration,  etc.  , 
and  the  elements  of  the  design  matrix  represent  the  levels  of  each  of  these 
stresses.  For  example,  x^  represents  the  second  level  of  the  stress  x^, 
etc.  The  column  titled  response  vector  represents  the  observed  response 
obtained  with  each  treatment  combination.  The  response,  y,  is  a contin- 
uous variable  such  as  arming  time  in  the  case  of  the  fuze,  or  possibly  depth 
of  penetration  in  the  case  of  a shaped  charge  warhead;  the  response,  y. 
for  the  ith  treatment  combination  may  be  expressed  as  a linear  combination 
of  the  treatment  levels  plus  some  random  error.  The  regression  model  and 
underlying  assumptions  are: 
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y:  n x 1 
u:  n x 1 


ysX1  |3  + u , 


is  the  observation  vector, 

is  the  vector  of  random  errors,  and  is  normally  distri- 
buted with  mean  vector  O and  covariance  matrix  <r^I,  i.  e.  , 
E{u)  = O,  E{u  u')  = cr^I, 


X:  m x n is  the  design  matrix  of  rank  r,  r < rri  n, 
(3:  m x 1 is  the  vector  of  regression  coefficients. 


A geometrical  interpretation  of  this  regression  model  and  its  relation  to 
performance  reliability  is  shown  in  Figure  2 ^Tg-bles  and  figures  can  be 
found  at  the  end  of  the  article.^  for  the  univariate  response  case,  (x^,  . . 

. . , x ) are  the  stress  variables,  (c, , . . . , c ) are  the  components  of  the 
m I m 

critical  reliability  boundary  vector  c.  The  points  shown  in  the(x^,  . . . , xj, 

plane  represent  the  treatment  combinations  for  the  regression  experiment, 
and  the  average  response,  y,  is  represented  by  the  regression  surface 
shown  above  the  points.  The  regression  equation  provides  estimates  of 
the  response  for  any  point  in  the  (x^,  . . . , x^)  space.  The  response  y at 

some  point  such  as  the  critical  reliability  boundary  c : m x 1 is  denoted 

(c)  2 

by  yv  'and  is  distributed  according  to  N(c*  (3,  c r ).  The  lower  performance 

(c)  (O) 

limit  which  the  response  y is  required  to  exceed  is  denoted  by  y » and 

consequently,  the  performance  reliability  R is  given  by: 

(1)  H(.)  = P[Y(C)  > y(0)}  = j7o>  "'V(C)|  C p.  e2)  dy(c)  ; *1<C'  »-  <=>  • 


where  n(x  , tr  ) is  the  density  function  for  a normal  population  with 

2 

mean  [1  and  variance  <r  . This  expression  represents  the  shaded  area 
under  the  normal  curve  shown  in  Figure  2. 

Thus,  our  problem  is  to  estimate  g which  is  a function  of  the  un- 
known parameters  (3,  cr^,  and  the  fixed  point  c,  based  upon  a sample  of 
size  n treated  as  a single  or  multiple  regression  experiment. 
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3.  2 Multivariate  Response.  The  univariate  model  will  now  be  ex- 
tended to  include  cases  where  more  than  one  continuous  response  may  be 
observed  on  a single  experimental  unit  such  as  S and  A arming  time, 
functioning  time,  and  self-destruct  time;  also,  the  responses  may  be 
correlated.  Multivariate  analysis  techniques  permit  the  correlation  be- 
tween responses  to  be  investigated.  As  before,  the  problem  is  best 
illustrated  by  examining  the  table  shown  in  Figure  3.:  The  design  ma- 
trix X is  exactly  the  same  as  for  the  univariate  case;  (x^,  . . . , x ) is 

still  the  vector  of  applied  stresses.  However,  instead  of  a single  re- 
sponse vector  of  y's,  we  now  have  p responses  (y^,  . . . , y ).  Thus, 

for  each  treatment  combination  we  observe  p responses  so  that  our 
response  vector  for  the  univariate  case  has  now  become  a response  ma- 
trix where  the  column  vectors  may  be  correlated,  and  the  rows  which 
represent  independent  response  vectors  are  uncorrelated.  The  multi- 
variate model  and  assumptions  are: 

Y = X'  B + U, 


Y: 

n x p 

is 

X: 

m x n 

is 

B; 

m x p 

is 

U: 

n x p 

is 

Uj’ 

j=l,  • • • 

. n 

the  response  matrix 

the  design  matrix  of  rank  r < m < n,  p < n-r, 
the  matrix  of  regression  coefficients, 
the  error  matrix, 

are  the  rows  of  U and  are  independently  and 
identically  distributed,  each  having  a p -variate 
normal  distribution  with  mean  vector  O and 
positive  definite  covariance  matrix  2. 


i c l 

From  the  multivariate  model,  a p xl  response  vector  y is  ob- 
tained for  the  response  at  the  critical  reliability  boundary  vector  c:  m x 1. 

(c) 

The  response  vector  y ' is  distributed  according  to  N(c*B,  2),  in  which 
the  p x p covariance  matrix  2 takes  into  account  any  correlations  be- 
tween responses. 


The  performance  reliability  R for  the  multiple  response  case  is 
given  by: 
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(2)  R(c)  = p(yJc)  > yj01 Y^c>  > y^j 


r... 

(O) 


y n(y(c)|c»  B,  2)  dy[c) 

/ ^ \ A 


= gp  (c‘  B,  S;  c) 


A graphical  representation  of  the  performance  reliability  in  two  di- 
mensions is  shown  in  Figure  4 for  the  multivariate  case.  The  above  in- 
tegral represents  the  volume  of  the  multivariate  normal  density  function 
over  the  shaded  quadrant  whose  vertex  y(O)  is  the  vector  of  specification 
limits. 

Thus,  the  general  problem  may  be  summarized  as  follows:  Based 
upon  the  results  of  a suitable  experimental  design  with  a sample  of  size  n, 
it  is  required  to  estimate  the  g function  for  the  univariate  and  multi- 
variate cases,  both  by  point  estimation  and  confidence  limits. 

4.  POINT  ESTIMATION-.  The  general  problem  is  to  estimate  the 
performance  reliability  functions  defined  for  the  univariate  and  multi- 
variate responses  both  by  a point  estimate  and  confidence  limits  based 
upon  responses  observed  on  a sample  of  size  n subjected  to  various  stress 
treatments  in  accordance  with  a suitable  experimental  design.  The  experi- 
mental designs  used  for  exploring  response  surfaces  [l,  2j  are  generally 
suitable  for  exploring  the  region  around  the  critical  reliability  boundary. 

4.1  Univariate  Point  Estimates.  The  g or  R functions  to  be  esti- 
mated may  be  written  as  follows  for  the  univariate  case: 


R(P,  <r2)  = g^c  1 p,  o-2;  c)  = 


(2tt)  ^2  exp  (-t2/2)  dt, 


( y^-c'P)/<r 
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where  c and  (3  are  m x 1 vectors. 

We  consider  three  types  of  point  estimates.  Suppose  we  write 

K(p,  <r)  = (y^  -c'p)/<r  , 

then 

(3)  R(P,  cr2)  = y (2tt.)  ^2  exp  (-t2/2)  dt. 

K(P,  cr) 


The  first  estimate  of  R is  based  on  using  K($,  a),  where  |3  and  ^ 
appropriate  estimates  of  (3  and  cr. 

The  least  squares  estimate  of  (3  is  given  by 
(4)  P = (XX')_1X  y , 

where  X:  m x n,  of  rank  m < n,  is  the  design  matrix,  and  y : n x 
the  response  vector.  An  unbiased  estimator  of  cr^  is 


(5) 


C2  (y-X'g)  ' (y-X'fr) 
n-m 


Thus,  we  may  use  the  estimate 

(6)  K(p,cr)  = (y^  - c'^/cr  , 
from  which  we  obtain 

(7)  R(P,  a-2)  = y (2tt)  1//2  exp  (-t2/2)  dt. 

K(|,  $) 


are 


1 is 


A second  estimate  is  based  on  the  UMVU  estimate  of  K,  namely 
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-julil 

"'f  r».  f-1 


f r(ii> 


* A. 

K(  p,  c)  , 


where  f = n-m  , from  which  we  may  use 


(9)  R(p,ir2)=  £ (2tt)  1//2  exp  (-t2/2)  dt 

K(p,cr) 


as  an  estimate  at  R(p,  cr  ) . 


Although  K is  a UMVU  estimate  of  K,  it  is  not  the  case  that  R 
is  a UMVU  estimate  of  R.  Consequently,  a third  procedure  is  based 
on  the  UMVU  estimate  of  R,  £4]  t and  is  given  by 


(lO)  R((3,(r)  = 


[o.njfiH  [£2  -1] 


B (T--T-) 


jll°l  Z..S1?).  > . atpA..— 

2 2ff*Nf(l-c'  (XX')c)  2 2^f(l-c‘(XX')"ic) 


— / 2 

Note  that  R(|3,  or  ) = 1 if  T\>1,  and  that  the  estimate  is  valid  for  criti- 
cal vectors  c such  that  c^XX1)  ^c  < 1. 

Unfortunately,  comparisons  of  the  risk  of  these  estimators  are  un- 
available, since  the  determination  of  the  variance  is  quite  complicated, 
and  was  not  attempted  in  this  paper. 
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4.  2 Multivariate  Case.  In  the  multivariate  case,  we  have 


,-l 


R(B;S)  = 


/» co  roo 


1(0)  * (O) 
yp 


-1/2  tr  Z"  (Y-X'B)'(Y-X'B) 

|2|pn/2(2»)pn/2 


dY, 


where  Y:  nxp,  X;  mxn,  B:  mxp.  As  in  the  univariate  case, 

we  can  consider^R{S,  Z)  as  an  estimate  of  R(B,  Z),  where  B =(XX')”*XY, 
and  Z = (Y  - X'B)*(Y  - X'B)  / [p(n  -m)3  . The  problem,  however,  is  still 
to  evaluate  the  multivariate  normal  distribution  over  an  orthant.  In 
fact,  whether  we  use  this  estimation  procedure  or  another,  the  diffi- 
culty of  carrying  out  such  an  integration  still  remains.  However,  for 
any  particular  problem,  one  can  employ  numerical  procedures  to  yield 
an  answer.  Another  possibility  which  has  not  been  considered  in  the 
literature  is  to  obtain  a lowe  r bound  for  R(B,  Z)  in  terms  of  known 
functions.  Further  work  in  this  area  is  required. 


5.  CONFIDENCE  INTERVALS.  The  problem  of  obtaining  confidence 
intervals  for  the  g or  R functions  is  considered  next.  The  general 
method  is  discussed  in  [33,  and  is  now  extended  to  the  regression  model. 
In  Section  4,  three  estimates  were  presented.  For  only  the  second  pro- 
cedure is  the  distribution  theory  known,  so  that  exact  confidence  inter- 
vals can  be  obtained.  However,  the  first  procedure  does  lead  to  approxi- 
mate or  asymptotic  intervals  based  on  the  normal  distribution. 

2 

5.1  Exact  Confidence  Intervals.  Since  R((3,  cr  ) is  a monotone 
function. of  K(p,  cf),  if  we  can  find  a confidence  interval  (K  , K ) for  K, 

X Lt 

we  will  then  have  a confidence  interval  (R^,  R^)  for  R,  where 


Ri  = y (2tt)  1 /2  exp(-y2/2)  dy. 

K. 

l 

It  is  shown  in  Appendix  A that  K((3,  cr)/  ||  a ||  - t(f,  6),  where  ||a||  = 

c'(XX')  ^c , has  a non-central  t-distribution  with  f = n-m  degrees  of 
freedom  and  non-centrality  parameter. 
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6 = ,(y^-  c'p)/(o-||a||)  = K((3,  <r)/||a  || 


Thus,  a lower  and  upper  confidence  limit  with  confidence  coefficient 
1 - a may  be  obtained  by  finding  the  values  of  6^  for  which 


(11)  P{t> 


) f,  6^}  = ‘v,  i = 1,  2,  where  = 1 - a/2, 


a = a/2.  Table  IV  in  [6]  may  be  used  to  obtain  6.  for  seventeen  valuei 


2 

of  c . 


The  tabulation  by  Resnikoff  and  Lieberman  [6]  of  the  percentage  points 
of  the  non-central  t- statistic  may  be  conveniently  used  to  obtain  the  limits 
6 and  6 that  satisfy  (11).  The  entries  in  the  table  give  the  values  of 


x such  that 


pQ~  > *}  ■ 


The  table  should  be  entered  for  the  degrees  of  freedom  f = n-m,  the 
probability  corresponding  to  a^,  and  x = K(|3,  cr)/(  Jj ajf^f)  • The 

required  non-c.cntrality  value  6 . =if  + 1 K , where  K is  the  stand - 

i P P 

ardized  normal  random  variable  exceeded  with  probability  p.  The  pre- 
sent concern  was  with  one  sided  tails  (one  sided  specification  limits)  for 
both  the  univariate  and  multivariate  cases.  A review  of  available  point 
and  confidence  methods  for  two  sided  tails  is  given  in  [V]  . 

5.2  Approximate  Confidence  Intervals.  If  we  expand  about 

2 

R(p,  tr  ),  we  obtain  the  result  that 

R(%aZ)  - R(p,o-2)~N(0,  V(p  l(r2))  , 


where 


V, (P.  «■)  = 0-2  [n(y^|c'p,  o-2)]  2 (c^XX1)"1  c + & |c  - ] . 


2tr2f 
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Consequently  (see  Appendix  B), 

~ N(Of:  1)  , 

from  which  we  obtain  the  confidence  interval 

|R(^.  tr2)  + , R(?,?2)  + zl.a/2^  V~(P.  »)  ], 


R(S.?2)  - 

ifw 


where  is  the  lOO  a % point  of  the  N(0:  l)  distribution. 

6.  SAMPLE  PROBLEM.  In  order  to  illustrate  the  results  of  the 
previous  sections,  a sample  problem  will  be  solved.  A model  repre- 
senting the  performance  of  a hypothetical  shaped  charge  warhead  section 
for  a missile  will  be  described,  and  the  performance  reliability  will  be 
evaluated  based  upon  a Monte  Carlo  simulation  of  test  results.  The  point 
estimates  and  confidence  intervals  obtained  using  the  methods  previously 
described  will  be  compared  with  the  true  value  of  the  reliability.  Only 
the  univariate  or  independent  response  cases  are  considered. 

6.1  Performance  Model.  The  warhead  section  to  be  evaluated  con- 
sists of  a shaped  charge  warhead  and  a Safety  and  Arming  Device.  It 
is  assumed  that  the  warhead  is  required  to  penetrate  at  least  lO  inches 
into  an  armor  plate  target  and  that  the  minimum  arming  time  of  the 
S and  A is  O.  5 seconds.  The  warhead  section  is  expected  to  meet  these 
performance  requirements  under  all  possible  combinations  of  vibration 
and  temperature  shock  that  may  be  encountered.  To  facilitate  the  illus- 
tration, only  two  stresses  are  considered  in  this  problem,  but  the  pro- 
cedure is  easily  extended  to  more  than  two  stress  variables. 

The  two  stresses,  vibration  in  g's  and  temperature  shock  in  stan- 
dard cycles,  are  denoted  by  X^  and  X^,  respectively.  Coded  levels 

of  the  stresses  are  used  throughout  this  problem  to  facilitate  the  analysis 
and  simulation  of  test  results.  The  relationship  between  the  coded  and 
actual  stress  units  is  of  no  importance  with  regard  to  illustrating  the 
reliability  evaluation  methods  and  will  be  disregarded. 
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The  critical  reliability  boundary  is  defined  by  the  vector 
c'  = (c^,  c , c~)  where  c. , c.  are  the  upper  stress  limits  specified 

for  vibration  and  temperature  shock,  respectively  and  c^  is  a dummy 

variable  required  to  make  the  vector  c consistent  with  the  design  ma- 
trix X and  is  equal  to  1.  The  coded  variables,  in  this  example,  are 
centered  on  the  critical  reliability  boundary  so  that  c'  = (1,  0,0). 

The  warhead  performance  is  measured  in  terms  of  depth  of  pene- 
tration t into  monolithic  armor,  and  S and  A performance  is  measured 
w 

by  arming  time  t^.  The  distribution  of  warhead  penetration  t and  arming 

time  t,  for  the  S and  A is  each  distributed  according  to  N [6^  + 0,  ,+0„ 

2 f O rlxl  2x2 

a 3*  The  true  values  of  the  parameters  are 


Warhead 


rl 

-O.  6 


1.  5" 


S and  A 


O. 6 sec. 


0.07 


0.03 


033  sec. 


These  models  thus  assume  that  the  average  penetration  decreases  linearly 
with  increasing  stress  and  that  the  average  arming  time  increases  very 
slowly  with  increased  stress  within  the  region  of  interest.  Thus,  byCl), 
we  see  that  the  performance  reliability  for  the  warhead  and  S and  A,  res- 
pectively, are 


WHD 


= P{t^>  lO"}  = 

V XXT  * 


l n 


(t'C  1 c’p,  cr  ) dt 


= g2  (c'P,  cr;  c)  = gx  (13",  1.5";  c)  =0.977 
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(13) 


R 


S and  A 


c 1 P,  cr  ) dt 


(c) 

f 


= gx  (c'(3,  a;  c)  = gj.  (O.  6,  . 033  ; c)  = O.  999  . 


The  performance  reliability  of  the  warhead  section  is  thus 


R ” RWHD  >RS  and  A 


. 976  . 


Dud  probabilities  were  not  considered  in  this  model.  The  evalu- 
ation of  dud  rates  requires  attribute  test  methods  which  are  not  as  effi- 
cient as  the  variables  plans  and  require  much  larger  sample  sizes.  In 
conducting  the  type  of  test  program  described  herein  an  estimate  of  the 
dud  reliability  may  be  made  by  noting  the  number  of  dud  failures.  How- 
ever, useful  interval  estimation  with  these  results  may  not  be  possible 
with  reasonable  confidence  coefficients.  When  a dud  occurs,  it  is  de- 
sirable to  repeat  the  appropriate  test  under  the  same  conditions  in 
order  to  avoid  or  minimize  having  to  work  with  missing  data  in  the 
test  plan. 

6.2.  Multiple  Regression  Analysis.  Multiple  linear  regression  ex- 
perimental designs  of  the  type  used  in  exploring  response  surfaces  were 
used  to  evaluate  the  performance  reliability  of  the  warhead  and  S and  A 
based  upon  the  stated  performance  model.  In  particular,  central  com- 
posite rotatable  experimental  designs  [.3]  were  used.  The  experiments 
were  conducted  with  sample  sizes  (n)  of  8 and  30  for  both  the  warhead 
and  S and  A.  The  treatment  combinations  and  the  responses  generated 
by  Monte  Carlo  simulation  of  the  performance  models  are  shown  in 
Tables  1 to  4. 

Least  Squares  estimates  of  the  regression  coefficients  and  error 
variance  were  made  for  the  test  results,  and  goodness-of-fit  tests  were 
conducted.  In  all  four  cases,  a linear  regression  model  was  found  to 
represent  the  data  adequately.  The  least  squares  estimates  of  the  re- 
gression coefficients  obtained  for  each  case  are  as  follows: 
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/V  /N  A. 


Item 

n 

Pi 

CO. 

Warhead 

IT 

13.  99 

-.175 

. 675 

30 

12.  75 

-.  975 

. 215 

S and  A 

8 

• 59 

.055 

.020 

30 

. 60 

,0705 

.0205 

Tests  of  significance  at  the  .05  level  performed  for  the  regress- 
ion coefficients  gave  the  following  results.  The  estimates  of  p and  p 

X Lt 

for  the  warhead  based  on  n = 8 were  not  significantly  different  from  zero. 
For  n = 30,  p^  was  not  significantly  different  from  zero,  but  p^  which 

corresponds  to  the  effect  of  the  vibration  stress  was  found  to  be  signi- 
ficantly different  from  zero,  which  corresponds  to  the  true  situation  for 
the  model.  In  the  case  of  the  S and  A, ^^(temperature  shock)  was  not 
significantly  different  from  zero,  and  p^  (vibration)  was  significantly 
different  from  zero  for  n=8  and  30. 

2 

Point  estimates  of  the  performance  reliability  R(p,  a ) at  c'=(l,0,0) 
were  made  using  the  UMVU  estimate  K(p,  a)  of  K(p,  cr).  Exact  one  sided 
lower  confidence  limits  using  the  non-central  t-distribution  were  also 
obtained.  A summary  of  these  results  is  tabulated  below,  and  a s ample 
computation  is  given  in  Appendix  C for  one  case.  Estimates  of  R(p,  o-2) 
based  on  the  estimates  K(^,  tr)  and  K(p,  cr)  are  also  included  in  the  appendix. 


Case 

Item 

n 

R 

R(.95) 

True  : 

1 

Warhead 

8 

. 974 

. 832 

. 977 

2 

30 

. 967 

. 905 

3 

S and  A 

8 

. 983 

. 891 

.999 

4 

30 

. 997 

. 981 

where  R ^is  the  Estimate  of  Performance  Reliability  based  on  the  UMVU 
estimate  K(p,  <r)  or  K(b,  <r)  and  R(.95)  is  the  one-sided  lower  95/&  con- 
ficence  limit  for  Performance  Reliability. 


487 


Design  of  Experiments 

A point  estimate  of  the  warhead  section:  reliability  is  given  by 


R = RWHD  ' RS  and  A * 


Conservative  . 90  confidence  intervals  for  the  warhead  section  reliability 
are  obtained  by  multiplying  the  lower  . 95  confidence  limits  for  the  war- 
head and  S and  A.  This  result  is  easily  proven  by  applying  the  Bonferroni 
inequality  to  obtain  a conservative  simultaneous  confidence  region  T 
for  Rwhd  and  Rg  A and  by  making  use  of  the  fact  that  the  product  1 

is  monotone  in  each  variable.  Thus,  we  obtain  the  following  results  for 
the  warhead  section  reliability. 


n 


R 


R(  > ■ 90) 


True  R 


8 

30 


. 957 
. 964 


. 741 
. 888 


. 976 


Design  of  Experiments 


REFERENCES 


1.  Box,  G.  E.  P.  and  Draper,  N.  R.  , "A  Basis  for  the  Selection  of  a 
Response  Surface  Design11,  J.  Amer.  Statist.  Assoc.  , 54  (1959)  622-654. 

2.  Box,  G.  E.  P.  and  Hunter,  J.  S.  , "Multifactor  Experimental  Designs 
for  Exploring  Response  Surfaces",  Ann,  Math.  Statist.  , 28  (1957)  195-241. 

3.  Einbinder,  S.  K.  and  Olkin,  I.  , "Reliability  Testing  and  Estimation 
for  Single  and  Multiple  Environments  (Preliminary  Report)",  Pro- 
ceedings of  the  Seventh  Conference  on  the  Design  of  Experiments  in 


Army  Research  Development  and  Testing,  U.  S.  Army  Research  Office, 


Report  No.  62-2,  August,  1962,  261-291. 

4.  Ghurye,  S.  G.  and  Olkin,  I.  , "Unbiased  Estimation  of  Probability 
Densities",  Abstract,  Annal s . Math . St ati s t ■ , 32  (1961)  1346. 

5.  Johnson,  N.  L.  and  Welch,  B.  L.  , "Applications  of  the  Non- 
central t-Distribution",  Biometrika,  31  (1940)  362-389. 

6.  Resnikoff,  George  J,  and  Lieberman,  Gerald  J.  , Tables  of  the  Non- 
central  t-Distribution,  Stanford  University  Press,  Stanford,  Califor- 
nia, 1957. 


489 


TABLE  X 


WARHEAD 


EXPERIMENTAL  DESIGN  N-8 


Sample 

M 

f2 

* w 

1 

•1 

-1 

13.3 

2 

41 

>1 

14.9 

3 

-1 

41 

16.6 

4 

41 

41 

14.3 

S 

0 

0 

12.6 

4 

0 

0 

12.7 

7 

0 

0 

12.2 

• 

0 

0 

15.3 

»1  • Vibration 

xz  • Temperature  Shock 

t^  • Penetration  (inches) 


TAB  LE  2 

WARHEAD  EXPERIMENTAL  DESIGN  Ns 30 


Sample 

fl 

w 

1 

-1 

.1 

13.8 

2 

+1 

-1 

12. 1 

3 

-1 

+1 

13.8 

4 

+ 1 

+1 

12.7 

5 

0 

0 

12. 1 

6 

0 

0 

10.9 

7 

0 

0 

’2.5 

8 

0 

0 

12.2 

9 

-1 

-1 

13.7 

10 

+1 

.1 

10.5 

11 

-1 

+1 

12.7 

12 

+1 

+1 

11.0 

13 

0 

0 

14.8 

14 

-1 

-1 

13.7 

15 

+1 

-1 

9.5 

16 

-1 

+1 

11.5 

17 

+1 

+1 

13.0 

18 

0 

0 

13.3 

19 

.1 

-1 

12.0 

20 

♦1 

.1 

12.5 

21 

-1 

+1 

14.4 

22 

+1 

+1 

11.4 

23 

0 

0 

13.9 

24 

-1 

.1 

14.8 

25 

+1 

.1 

11.5 

26 

-1 

+1 

15.6 

27 

+1 

+1 

14.3 

28 

0 

0 

12.0 

29 

0 

0 

16.0 

30 

0 

0 

12.3 

*1  ■ Vibration 


*2  ■ Temperature  Shock 
tw  • Penetration  (inches) 


TABLE  3 


S It  A EXPERIMENTAL  DESIGN  N=8 


Sample 

a 

^2 

Arming  Time 
(seconds) 

1 

-i 

-1 

.48 

2 

+i 

-1 

.63 

3 

-i 

41 

.56 

4 

41 

41 

.63 

5 

0 

0 

>61 

6 

0 

0 

.56 

7 

0 

0 

.60 

8 

0 

0 

.64 

*1  ■ Vibration 

*2  » Temperature  Shock 


TABLE  4 


S It  A EXPERIMENTAL  DESIGN  N»30 


Arming  Time 


Sample 

li 

(seconds) 

1 

-1 

-1 

.55 

2 

41 

-1 

.63 

3 

-1 

41 

.56 

4 

41 

41 

.71 

5 

0 

0 

.57 

6 

0 

0 

.68 

7 

0 

0 

.67 

8 

0 

0 

.60 

9 

.1 

.1 

.50 

10 

41 

-1 

. 66 

11 

.1 

41 

.53 

12 

41 

41 

.73 

13 

0 

0 

.57 

14 

-1 

• 1 

.58 

15 

41 

• 1 

.66 

16 

-1 

41 

.57 

17 

41 

41 

.72 

18 

0 

0 

.55 

19 

-1 

-1 

.46 

20 

41 

-1 

.64 

21 

-1 

41 

.54 

22 

41 

41 

.65 

23 

0 

0 

.59 

24 

.1 

.1 

.51 

25 

41 

-1 

.67 

26 

-1 

41 

.56 

27 

41 

41 

.70 

28 

0 

0 

.53 

29 

0 

0 

.61 

30 

0 

0 

.60 

*1  • Vibration 

*2  ■ Temperature  Shock 


UNIVARIATE  RESPONSE 
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are  independently  and  identically  distributed  with  mean 


PERFORMANCE  RELIABILITY 
UNIVARIATE  RESPONSE 


MULTIVARIATE  RESPONSE 


u'^  , ...  , uy  are  Independently  identically  distributed 
common  p x p positive  definite  covariance  matrix  r . 


PERFORMANCE  RELIABILITY 
MULTIVARIATE  RESPONSE  (VECTOR) 


n(c'  B,  z)  dy 
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Define  K = (y^-  c '$)/a  and  K(3,g)  = (y^-  c'p)/o.  We  first 
note  that  £ and  a are  independently  distributed.  Since  EfS  = 
we  have  that  E(y^-  c'£)  = (y^-  c'p).  Also  v = fa2/ a2  has  a 
distribution,  f = n - m,  so  that 


rffi) 

^ r(|) 


Hence, 


r(J) 

E — 


r(^i) 


1 

a 


^ r<f  > 


which  proves  that  K(0,a)  = — — 


/v  ^ 

K(P,a)  is  an  unbiased  estimator 


v?  r(^i) 

of  K(P,a).  By  completeness,  it  then  follows  that  K is  the  unique  such 
estimator,  and  hence  is  UMVU. 

An  alternative  approach  is  also  useful,  namely,  that  K(p,a)/||a||  s t(f,6) 
has  a non-central  t-distribution  with  f degrees  of  freedom  and  non- 
centrality parameter 


8 = 


- y 


(o). 


c'P  _ K(Pj a) 


a a 


2 -1 

where  ||a||  = c'(XX')  c.  To  see  this,  we  write 


,A  A % 

K(P,<T) 


y^-  c*p  _ c'^-c'p 
Vyar(c'f)  Vyarfc'p) 


A 

a 


vVar(c^) 


But  c'p  = c'  (XX')"1Xy,  and  hence  var(c'$)  = c2  c^XX'J'^c  s c2  ||a| 
Thus, 


» A A„ 


y(°)-  c«p  _ c^-c'p 


K(Ml 

Hall 


a a 


o a 


A 

a 


By  noting  that  Et(f,6)  «=  6 Jf] 2 r(^i)/r(|),  ve  can  also  obtain 


E K(P,o)  = K(P,a). 
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Since  B($,cP)  is  a function  of  the  sample  moments,  it  follows 

A a2  2 

that  B(p,or  ) is  asymptotically  normal  with  mean  R(p,cr  ) and  variance 


zb  _\v«r(°2)  ■ 


*°2WJ 


The  cross-product  terms  involving  p^  and  drop  out  because  of 

A a2 

the  independence  of  p and  a . Prom 


R(b,s2) 


/ (Sn)”1/2  exp (-1/2  t2)  dt  , 


(y^-  c'b)/s 


we  obtain 
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dR 

*1 


Cl  exp[-i/2.(y(0):c'b)2]. 


s 


s 


8r  (y^^-c'b)  1 

as2  2S2  & S 


(0) 


exp[-l/2^-i--g'b)  } 


Also,  Cov(p±,pj ) = c2  a , where  A = (XX')"1,  Var(£2)  =2 ak/t,  and 

hence  the  asymptotic  variance  is 


V,(P,o2)  » o2[n(y(0) 


c'pj  02)]2  (c'(XX')"1  c 


, (y(0)  - cp)2 
+ r<F^  J 


But,  V.&S2)  is  a rational  function  of  the  sample  moments,  so  that, 

A ^2  p 

by  Slutsky's  Theorem,  Vo(p,o  ) converges  in  probability  to  Vo(p,0  ), 
and  hence 


gfe-°2-l  ■-  R(P>°2)  - H(0,  1)  . 


APPENDIX  C 


The  computation  of  the  point  and  confidence  interval  estimates  for 
the  performance  reliability  of  the  warhead  for  the  sample  size  n = 8 
is  described  in  this  appendix.  Prom  the  test  data  in  Table  I,  we 
obtain  the  following  results: 

Point  Estimation 


o . 2.958,  c' 


(1,  0,  0),  f = n - m = 8 - 3 = 5 


K (p,o) 


y(0)-C'£ 

A 

0 


10  - 13.99 
1.720 


- 2.320 


- 1.95 
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Substituting,  these  two  estimates  of  K(0/r)  in 

R O,*2)  = I (2n)'  /2e‘y2/2  dy 
K(p,o) 

» A.  «.  . 

gives  the  two  estimates  of  reliability  R(S,o)  and  R(p,a)  , 
respectively . Thus , 


R(P,ff)  - 

00 

/ 

dy  » 

.9898  f and 

- 2.520 

R(P,a)  = 

00 

/ 

(2,)-X-ya/2 

dy  = 

•974. 

- 1.95 

Confidence  Intervals 

||a||2  = c'fXX*)"^  * | for  c'  « (1,  0,  0)  . 

Following  the  notation  of  Resnikoff  and  Lieberman,  a confidence 
interval  may  be  obtained  using  the  non- central  t- tables  in  [6].  The 
percentage  points  of  t are  denoted,  by  x(f,6,e)  where  x is  the 
value  such  that  P > x | f,6}  = e . 

ft 

X . ZiPiS)  ■ (-2.320)  . - 2.935. 

* M 

The  one  sided  lower  .95  confidence  limit  for  R is  obtained  by 
finding  the  corresponding  limit  for  K(P,o)  because  of  the  monotone 
relation  between  R and  K(0,o).  The  1-a  confidence  limit  for 
K(fJ,or)  is  obtained  by  solving 


■ xfr.^^.seg 


> 


501 


x(5,  5>95,  .95)  = - 2.935  • 

Making  use  of  the  relation  x(f,B,e)  = -x(f,-6,l-e),  we  obtain 

x(5,  -&>95,  '.05)  = 2.935  • 


From  the  Resnikoff-Lieberman  table  of  percentage  points  of  t,  we 

obtain  a non- centrality  value  6 = 7f  + l k = 76  (1.107)  = 2.712 

P 

by  interpolating  on  K . Since  6 = p)-  the  .95  lower  confi- 

P M 

dence  limit  for  K(p,  a)  is 


K(P,cx) 

•95 


&.95  = -2.712 
7 n 7 8 


-.959 


Finally,  the  .95  confidence  limit  for  R is 


R (P,02)  = / (2*)"  /2e“y  /2  dy  = .832  . 

.95  K(P, a) 

• 95 


6 Q(.  may  also  be  computed  using  the  Johns on-Welch  table  IV  and 
following  the  procedure  on  page  372  of  [5]. 
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